In this paper, we point out that suboptimal noise-data mapping leads to slow training of diffusion models.

We study deep neural networks (DNNs) trained on natural image data with entirely random labels. Despite its popularity in the literature, where it is often used to study memorization, generalization, and other phenomena, little is known about what DNNs learn in this setting. In this paper, we show analytically for convolutional and fully connected networks that an alignment between the principal components of network parameters and data takes place when training with random labels. We study this alignment effect by investigating neural networks pre-trained on randomly labelled image data and subsequently fine-tuned on disjoint datasets with random or real labels. We show how this alignment produces a positive transfer: networks pre-trained with random labels train faster downstream compared to training from scratch even after accounting for simple effects, such as weight scaling. We analyze how competing effects, such as specialization at later layers, may hide the positive transfer. These effects are studied in several network architectures, including VGG16 and ResNet18, on CIFAR10 and ImageNet.

We compare the performance of randomized classical and quantum neural networks (NNs) as well as classical and quantum-classical hybrid convolutional neural networks (CNNs) for the task of supervised binary image classification. We keep the employed quantum circuits compatible with near-term quantum devices and use two distinct methodologies: applying randomized NNs on dimensionality-reduced data and applying CNNs to full image data. We evaluate these approaches on three fully-classical data sets of increasing complexity: an artificial hypercube data set, MNIST handwritten digits and industrial images. Our central goal is to shed more light on how quantum and classical models perform for various binary classification tasks and on what defines a good quantum model. Our study involves a correlation analysis between classification accuracy and quantum model hyperparameters, and an analysis on the role of entanglement in quantum models, as well as on the impact of initial training parameters. We find classical and quantum-classical hybrid models achieve statistically-equivalent classification accuracies across most data sets with no approach consistently outperforming the other. Interestingly, we observe that quantum NNs show lower variance with respect to initial training parameters and that the role of entanglement is nuanced. While incorporating entangling gates seems advantageous, we also observe the (optimizable) entangling power not to be correlated with model performance. We also observe an inverse proportionality between the number of entangling gates and the average gate entangling power. Our study provides an industry perspective on quantum machine learning for binary image classification tasks, highlighting both limitations and potential avenues for further research in quantum circuit design, entanglement utilization, and model transferability across varied applications.

We thank all reviewers for their useful comments. Table 1: AUCROC obtained from Likelihood Regret on Glow and PixelCNN. We now address detailed concerns of each reviewer. We apologize for the confusion caused by the notation. Please refer to items 1-3 for concerns regarding why we focus on V AE's OOD detection.

This is indicated in Tab. 2 (see Sec. 6.3) by the low correlation (

Deep state-space models (DSSMs) enable temporal predictions by learning the underlying dynamics of observed sequence data. They are often trained by maximising the evidence lower bound.

Data condensation techniques aim to synthesize a compact dataset from a larger one to enable efficient model training, yet while successful in unimodal settings, they often fail in multimodal scenarios where preserving intricate inter-modal dependencies is crucial. To address this, we introduce ImageBindDC, a novel data condensation framework operating within the unified feature space of ImageBind. Our approach moves beyond conventional distribution-matching by employing a powerful Characteristic Function (CF) loss, which operates in the Fourier domain to facilitate a more precise statistical alignment via exact infinite moment matching. We design our objective to enforce three critical levels of distributional consistency: (i) uni-modal alignment, which matches the statistical properties of synthetic and real data within each modality; (ii) cross-modal alignment, which preserves pairwise semantics by matching the distributions of hybrid real-synthetic data pairs; and (iii) joint-modal alignment, which captures the complete multivariate data structure by aligning the joint distribution of real data pairs with their synthetic counterparts. Extensive experiments highlight the effectiveness of ImageBindDC: on the NYU-v2 dataset, a model trained on just 5 condensed datapoints per class achieves lossless performance comparable to one trained on the full dataset, achieving a new state-of-the-art with an 8.2\% absolute improvement over the previous best method and more than 4$\times$ less condensation time.